How do language models segment their internal experience of the world of words to progressively learn to interact with it more efficiently? This study in the neuropsychology of artificial intelligence investigates the phenomenon of synthetic categorical restructuring, a process through which each successive perceptron neural layer abstracts and combines relevant categorical sub-dimensions from the thought categories of its previous layer. This process shapes new, even more efficient categories for analyzing and processing the synthetic system's own experience of the linguistic external world to which it is exposed. Our genetic neuron viewer, associated with this study, allows visualization of the synthetic categorical restructuring phenomenon occurring during the transition from perceptron layer 0 to 1 in GPT2-XL.

The result of this categorical segmentation is reflected in the creation, by each neuron, of a synthetic category of thought, a concept, or, to put it differently, a categorical dimension carried by this neuron [101, 102]. This synthetic conceptual category is, among other things, defined by its extension, that is, the set of tokens for which the neuron associated with this category is (sufficiently) activated. In a previous work [105], we investigated the mathematical-cognitive factors of categorical segmentation performed by the synthetic neurons of language models. In this preliminary exploratory study, we examined, both quantitatively and qualitatively, the genetic elements influencing this categorical segmentation. Based on the aggregation function 1 Σ(w i,jx i,j) + b, which partly governs this cognitive process, we identified three key causal elements of a mathematical and cognitive nature involved in this conceptual partitioning process. First, the "x effect" or synthetic categorical priming, which refers to the fact that the activation of the categories carried by precursor neurons in layer n affects the activation of the categories specific to their associated target neurons in layer n + 1, thereby directly impacting their categorical extension. In other words, the more a token belongs to the extension of a precursor category in layer n (i.e., the more this token is activated in the involved neuron), the greater its potential to belong to the extension of its superordinate category (i.e., its potential activation in the neuron of layer n +1). This phenomenon of categorical priming thus partly governs the categorical segmentation performed in layer n + 1, that is, the determination of the subset of tokens constituting the categorical extension of the concepts carried by the neurons in layer n + 1. Second, the "w effect" or synthetic categorical attention, which relates to the fact that the weights of the connections between a target neuron (layer n + 1) and its precursor neurons (layer n) govern the degree of relevance attributed to the precursor categories in constructing the categorical segment of their corresponding target neurons.

How do the synthetic neurons in language models create "thought categories" to segment and analyze their informational environment? What are the cognitive characteristics, at the very level of formal neurons, of this artificial categorical thought? Based on the mathematical nature of algebraic operations inherent to neuronal aggregation functions, we attempt to identify mathematico-cognitive factors that genetically shape the categorical reconstruction of the informational world faced by artificial cognition. This study explores these concepts through the notions of priming, attention, and categorical phasing.